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Description 

[0001] This invention relates to techniques lor the measurement of object height and volume, more particularly to 
measuring heights of features on a printed circuit board by optical imaging. Hybrid confbcal microscopy may be 
5 employed. 

[0002] There are many applications that require measurement of height and volume using non-contact means. For 
example, the very first step in a surlace-mount printed circuit board assembly process is the printing of solder paste 
onto the solder pads. In this case, the amount of solder paste deposition onto the solder pad is crucial to the integrity 
of the assembled printed circuit board. A wrong amount of solder will cause poor solder joint. Excessive solder paste 
10 will cause bridging between adjacent pads. Less solder paste will cause poor contact, in another example, the wire 
bonding process connects a wire from the bond pad of a semiconductor die to a lead of a lead frame. This process 
requires the measurement of the wire loop height and the ball height. 

[0003] It can be observed that most of the existing on-line methods of measuring the height and volume of object such 
as solder paste rety on sampling of several points on the deposition. Sampling measurement does not provide sufficient 
IS data and information for effective process control. On the other hand, detailed measurement which covers the entire 
solder paste deposition is time consuming which can only be carried out off-line. In the case of measuring the wire loop 
and ball height in the wire bonding application, manual techniques relying on focusing are commonly used. This tech- 
nique is difficult to be automated and is slow and is in nature. 

[0(K)4] Laser structured light and confocal scanning laser microscopy approaches are the two common methods for 
20 the measurement of object height and volume. The laser structured light approach makes use of a laser line projector 
to project a finely focused light strip over the solder paste. This displacement of the light strip from its original axis is 
observed. Using triangulation principle, the displacement of the light strip is used to calculated the height of an object. 
[0(K)5] In the method of scanning confocal light-optical microscopy, the focusing of laser beams can be measured to 
determine the height of object. The entire solder paste deposition can be measured using the scanning confocal light- 
25 optical microscopy method. However, this method is time consuming and can only be applied off-line. 

Fig. 1 The block diagram of the present invention. 

Fig. 2 An Illustration of the depth of field effect associated with an optical system using a thin lens model. 
Fig. 3 The multiple imaging paths of the optical system focused at different height planes in the object space. 
30 Fig 4. Feature extraction using threshold-intensity-transform method. 

Fig 5. Flow chart illustrating the operational procedure of the present Invention. 

Rg 6. An example of the determination of the best-fbcal-plane for an object feature Is shown in Fig 6A. 

[0006] According to the present Invention, a method is provided that uses multiple Imaging paths, with each path set 
35 to a different Image distance, to acquire multiple images of the object through a common field of view. An image sensor 
is attached to each of the imaging paths to acquire images of the object. The multiple imaging paths are set to different 
Image distance and hence the corresponding object distance is different from one another. Images appearing on the 
image sensors attached at different Imaging paths are focused at different heights in the object space. A con^esponding 
number of height planes are created in the object space by the multiple Imaging paths. Object features having different 
40 heights will appear with different degree-of-fbcus at a particular Imaging path. Object features that are located within the 
preset objective distance of a particular Imaging path will appear focused on that image sensor. According to the 
present Invention, the height of an object feature is computed according to the degree-of-focus of the Images acquired 
by the different imaging paths. Image processing techniques are used to measure the degree-of-fbcus of an Image 
acquired by a frame gralDber from the image sensors of the multiple imaging paths. The variable degree-of-focus gath- 
45 ered from the multiple image paths and sensors are used to compute the height and the volume of the features of the 
object. 

[0007] The present Inventions also disclosed a embodiment of an apparatus having optics. Image processor and X- 
Y scanner. The optics disclosed in the invention Includes multiple Imaging paths created by the use of multiple beam 
splitters with a common objective lens. The objective lens is positioned In Isetween the object and the multiple beam 

so splitters. The multiple beam splitter splits the Image into multiple paths. Each of the Imaging path is attached with an 
Image sensor such as a charge coupled device CCD sensor. The multiple image sensors of the optica! means are con- 
nected to the image processor in which frame grabbers are used to digitize the images for image processing and anal- 
ysis. Image processing techniques are applied to the images to identify the features of Interest, determine the degree- 
of-focus on each of the images, compute the height Information and make decision on acceptance or rejection accord- 

55 ing to pre-stored criteria. The XY scanner is used in conjunction with the optics and Image processor to cover a large 
area. The X-Y scanner will either move the optics of the present Invention or the object so that a area larger than a sin- 
gle f ield-of-view is covered. 

[0008] As shown in Figure 1 , one embodiment of the present invention includes an objective lens 1 and multiple beam 
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splitters 2, 4 and 1 1 . to form multiple imaging paths 3, 7, 9 and 12; each of the imaging paths has an image sensor 5. 
8. 10, and 13. The image sensors 5, 8, 10 and 13 are connected to a frame grabber 14 so that the respective images 
can be read and processed by an image processor such as a host computer 15. Each of the imaging paths 3, 7, 9, and 
12 is predetermined to focus at a different object distance on the common field of view 16 containing the object 6. A 
5 con-esponding number of height planes 31 , 32, 33, and 34 (see Figure 3) are created by the multiple imaging paths 3, 
7. 9 and 12 in the object space. The objective lens 1 . the multiple beam splitters, and the multiple image sensors are 
enclosed in a light shield casing collectively known as the optical system 200. 

[0009] Images 40. 41 . 42 and 43 (see Figure 6) acquired from the image sensors 5. 8, 10 and 1 3 attached to the mul- 
tiple imaging paths 3, 7, 9. and 12 are digitized by the frame grabber 14 and communicated to the host computer 15. 

10 The images 40. 41 . 42 and 43 are subject to image intensity correction as each of the images has different intensity due 
to the effect of the beam splitters 2, 4 and 1 1 . A beam splitter splits an incoming image into two paths, hence reducing 
the intensity of the splitted images. The image intensity correction method equalizes the mean intensity of the different 
images to a nominal value. Noise removal techniques such as the threshold-intensity-transform method can be applied 
to remove the background noise. Object features 45 are then evaluated after the images have been conrected and noise 

IS removed. 

[0010] In Figure 1 , the object 6 is being placed on a X-Y scanning mechanism 202 such that it can be nx>ved so as 
to position different parts of the object 6 under the objective lens 1 of the optical system 200. Alternatively, the X-Y scan- 
ning mechanism 202 can also be attached to the optical system 200 while the object 6 remains stationary. The optical 
system 200 is then moved by the X-Y scanning mechanism 202 to scan the object 6. 
20 [0011] The degree-of<fbcus of an object feature 45 is computed for all the images 40. 41 . 42 and 43 (see Figure 6) 
acquired from the multiple imaging paths 3. 7. 9 and 12. An object feature 45 will appear more focused on one of the 
images 40, 41 , 42 and 43 than the others. The height of the object feature 45 is classified according to the height plane 
of the imaging path that it is focused. This procedure is applied to all the object features and hence the height of all fea- 
tures are computed. 

25 [0012] An exemplary design of the optical system used for the present invention is disclosed. The objective lens 1 is 
designed for finite conjugate imaging and is properly corrected for distortion at a selected band of wavelength. A thin 
lens model is used to nrKxjel the objective lens 1 . 
The basic lens formula is given by 

30 ^ ^ .| 

where is the object distance 31 of the first imaging path 3: and v^ is the con-esponding image distance; / is the effec- 
35 tive focal length of the objective lens 1 . 
[001 3] Rearranging equation (1) gives 

^1 = ir;^^ (2) 



40 



45 



55 



[0014] The second imaging path 7 is focused at an object distance - 5, 32; 5 Is the height resolution and V2 is the 
corresponding image distance of the second imaging path 7 as follows: 



[0015] For the n^ imaging path, the object distance is u ^ - (n-1)5 ; and is the image distance of the n^ imaging 
so path. 

" u,-(n-1)6-f W 

[001 6] Using equation (4), the image sensors for the respectively image paths are positioned at v^. Suitable compen- 
sation must also be applied to the image distance due to the different length of media Some of the imaging paths con- 
sists of more glass media than air. 
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[0017] Let /i. /2. /3..../nbetheimages40,41.42arKl43acxiuiredbythesensor5 5, 8, 10and 13 located at the dif- 
ferent imaging paths 3, 7, 9 and 1 2 respectively. For image features whose height are between 1/^+6/2 and - 6/2 , 
they will appear focused on image 1^. Similarly, image features with height between a^- (/7-1)5 + 5/2 and 
u^- (r7-1)5 - 5/2 will appear focused on image /„. 
5 [0018] As shown in Figure 2, resolution of height measurable by the present invention is related to the depth-of-f ield 
of the optical system. The depth-of-f ield 20. D. of an optical system is given by 



where u = object distance 24, a = diameter of aperture 21 , f = effective focal length, and c = circle of least confusion 
23. Fig. 2 Illustrates the depth-of-f Ield effect of an optical system involving a thin lens model. The circle of least confu- 
sion 23. c, is dictated by the pixel size of the image sensor 5. 8. 10 & 13. The desired depth-of-f ield 20. D, of the optical 
15 system can be designed by varying the diameter of the aperture 21 , a. and the effective focal length, f, of the objective 
lens 1. 

[001 9] The object features contained in the image consisting of the object are extracted by various image processing 
techniques. For those sidlled in the art, image processing techniques such as region segmentation, edge detection and 
labelling can be used for feature extraction. An example of using the threshold-intensity-transform method is described 
20 hereinafter. The intensity of a pixel in the transformed image, /out* ^^^^ intensity in the original image, 

is less than the threshold. T; while pixel intensities larger than the threshold are preserved in the transformed image. 
Mathematically, the transformed image /out which consists of the extracted features are expressed in the following 



[0020] The threshold-intensity-transfbrm method is illustrated in Fig 4. The threshold T is partially determined by the 
background noise level of the image and can be manually selected for a particular imaging system. In the threshold- 
30 intensity-transform image, pixels with zero intensity are treated as background while non-zero intensity pixels forming 
clusters are regarded as object features 45. 

[0021] According to one aspect of the present invention, the best focal plane of a particular object feature 45 can be 
determined to determine the height of the object. For those skilled in the art of image processing, there are many ways 
of determining whether an object feature is focused on the image plane. References ( Eric Krotkov, "Focusing", Inter- 
35 national Journal of Computer Vision, Vol 1 , page 223-237. 1987; and J.R Schlag, A.C. Sanderson, Q.P. Neumann, FC. 
Wimberly, "Implemention of /Vutomatic Focusing /Mgorithms for AComputer vision System with Camera Control," Carn- 
egie Mellon University Technical Report, CMU-R1-TR-83-14, 1983) give good accounts of the image processing tech- 
niques for focus determination. These techniques can be used in the present invention. 

[0022] An exemplary technique of determining the degree-of-focus of object features is disclosed hereinafter. For the 
40 sake of clarity, the solder paste is used as the object for illustration purpose. If an object feature of the object 6 is 
focused, the area of the object feature 45 is smallest and its intensity energy is highest. The area of the object feature 
45 is computed by summing the total number of pixel that has non-zero intensity forming a cluster. The total intensity 
energy of the object feature 45 is calculated by summing all the intensity of pixels that belong to the object feature. As 
there are multiple imaging paths 3, 7, 9 & 12 forming multiple images 40, 41, 42 & 43, these two parameters, i.e. area 
45 and intensity energy, are computed for all the images /i , I2, /s.-.- In ^0, 41 . 42 and 43. An object feature 45 is considered 
focused on image In when its area is the smallest and its intensity energy is highest on the n^ image plane. 
[0023] The computation of the area and intensity energy of an object feature 45. labelled /c. can be formulated in the 
following: 



10 




(5) 




25 



50 




1 if /n(x,y)>r 
0 if /n(x,y)<r 



(6) 



55 
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s(;c.n)= 'n(^.y) (7) 

xyek 

where A(k,n) is the area o1 an object feature k in image /p. 

S(Kn) is the intensity energy of an object feature k in image /„. 

Best'focal-plane (k) = min {A(k,n)] & max {S(Kn)] for all n images. (8) 

[0024] Fig. 6 illustrates an example for the determination of the best-focal-plane of an object feature. In this particular 
example, four images 40, 41 . 42 and 43 are acquired from the respective sensors of the four imaging paths. An object 
feature 45 is found to be focused on image 1^ using equation (8). Ttie height of this object feature 45 is therefore approx- 
imated to U3. 

[0025] The overall procedure for measuring the height and calculating the volume of object features is illustrated in 
Fig. 5. Step 98 is the beginning of the procedure. At step 100, images are acquired into the host processor 15 memory. 
At step 102. pre-processing such as noise removal algorithms and intensity normalisation are applied. One such algo- 
rithm has been mentioned earlier is the threshold-intensity-transform method. At step 1 04, feature extraction algorithms 
such as edge detection is applied to detect the edges of object features 45. At step 106. the degree-of-focus of the 
object features 45 are computed for all the different images 40, 41, 42 & 43 acquired from the different imaging paths 
3, 7, 9 & 1 2. At step 1 08, the degree-of-focus values from the different images 40, 41 , 42 & 43 are used to compute the 
height each of the features 45. At step 110, volume of the object 6 can be computed. 

[0026] The above procedure can be repeated for different areas of the object 6 when the object 6 is larger than the 
field of view of the optical system 200. Either the optical system 200 or the object 6 can be moved by the X-Y scanning 
mechanism 202. Hence, the height of a large area of the object 6 can be measured using the present invention. 
[0027] It is noted that in the present invention, the light for imaging need not be a laser. Further, as shown in Figure 

1 . no pinhole is needed to block any of the light that travels toward the imaging sensor (e.g., CCD). In this way. a rela- 
tively optically and mechanically simple apparatus can be used to determine the height of features on objects such as 
printed circuit board assemblies. 

Claims 

1 - An apparatus for determining topographical variation on the surface of an object 6, comprising: 

lens 1 for directing ot)iect light passing from the surface of the object 6 to a beam splitting system 2&4&1 1 , for 
focusing to a distance farther than the beam splitter system 2&4&1 1 ; 

the beam splitter system 2&4&1 1, for splitting light directed by the lens 1 from the surface to obtain multiple 
beam paths, the beam splitter system 2&4&1 1 comprising a plurality of beam splitters 2, 4. 1 1 to split the object 
light passing from the surface and the lens 1 repeatedly to obtain the multiple light paths, each beam splitter 2. 
4. 1 1 splitting incoming light thereto into branches; 

a plurality of light detectors 5, 8, 10, 13 each sensing light transmitted on a different light path from the beam 
splitter system 2&4&1 1 to form an image of the surface, the light paths having different path lengths from the 
lens 1 to the light detectors 5. 8, 10. 13 to achieve different degree of focus of the image on the light detectors 
5, 8, 10. 13; and 

processor 15 for determining topographical values of the surface of the object 6 based on the difference in 
degree of focus on the light detectors 5. 8. 10, 13. 

2. An apparatus according to claim i wherein the beam splitters 2&4&1 1 are arranged as a cascade so that a beam 
splitter next in order in the cascade relative to another beam splitter splits the light passing from the another beam 
splitter. 

3. An apparatus according to any of claims 1 to 2 wherein the light detectors 5. 8. 10, 13 each has multiple pixels. 

4. An apparatus according to any of claims 1 to 3 wherein the light detectors 5. 8, 10, 13 are charge coupled devices 
(CCD). 

5. An apparatus according to any of claims 1 to 4 comprising a light source for illumining the surface. 

6. An apparatus according to any of claims 1 to 5 conprising a light source that illumines an area of the surface that 
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has a plurality of features that rise above a plane such that the features can be illumined simultaneously sind the 
processor can determine variation in height of the features. 

7. An apparatus according to any of claims 1 to 6 wherein no pinhole is employed to cut off part of the image from the 
light detectors 5, 8, 10. 12. 

8. An apparatus according to any of claims 1 to 8 wherein the light detectors 5, 8, 10, 12 each include multiple pixels 
to detect variation in the image from the corresponding light path and wherein the processor 1 5 determines the var- 
iation within each image detected by a light detector 5. 8. 10. 12 and determines the topographical values of the 
surface based on the variation within each image as well as among the images detected by the light detectors 5, 8, 
10, 12, 

9. A method for determing topographical variation on the surface of an object comprising: 

directing object light passing from the surface of the object 6 as a field of view to be split into different light paths 
while focusing the same field of view to a distance farther than a beam splitter system 2&4&1 1 used for splitting 
the beams; 

with the beam splitting system 2&4&1 1 splitting repeatedly light passing from the surface associated with the 
field of view to obtain multiple beam paths; 

sensing light transmitted on each different light path to form an image of the field of view of the surface, the light 
paths having different path lengths due to the focusing on light detectors 5, 8. 10, 12 used for the sensing to 
achie\/e different degree of focus of the image; and 

determining topographical values of the surface of the object 6 based on the difference in degree of focus on 
the light detectors 5. 8, 10. 12. 

10. A method according to claim 9 comprising detecting difference in variation in the image from each light path and 
calculating the variation in the image in that light path to determine the topographical values of the surface based 
on the variation within each image as well as among the images. 
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